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Crowdsourcing  and  Emergency 

Response 


•  Real-time  citizen  interaction  is  transforming  crisis  response 


► 


► 


► 


Haitian  citizens  collaborated  with 
volunteers  worldwide  to  map  damage 
during  2010  earthquake 

Social  media  figured  prominently  in 
government  response  to  Hurricane  Irene 

“Social  media  follow  Hurricane  Sandy's 
destructive  path”  -  USA  Today 


►  In  Boston  Marathon  bombing,  Boston  PD  used  Twitter  to  monitor 
public  reaction,  engage  public,  correct  rumors,  assist  in 
identifying/locating  suspects 


Social  media  (e.g.,  Facebook,  Twitter)  provide  natural  avenue 
for  citizen  engagement  with  law  enforcement  /  emergency 
rnanagement  during  a  crisis 
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Boston  Police  Dept.  O 

©bostonpolice 


Boston  Marathon 


^  Follow  ^ 


Boston  Police  looking  for  video  of  the  finish  line 
#tweetfromthebeat  via  @CherylFiandaca 


The  Boston  Globe  O 

©BostonGlobe 


^  Follow 


BREAKING  NEWS:  Two  powerful  explosions  detonated 
in  ( I'  ‘  ‘ 


It  @mviser  W 
administratio: 
necessary  in  ti 

3:52  PM -15  Apr  20 
98  RETWEETS  8  FAVO 
OBOStO 


Boston  Police  Dept.  $ 

©bostonpolice 


CAPTURED H!  The  hunt  is  over.  The  search  is  done.  The 
terror  is  over.  And  justice  has  won.  Suspect  in  custody. 

8:58  PM -19  Apr  2013 


j.fbi.gov  or  1- 


^  ir 


135^7  RETWEETS  45,944  FAVORITES 


#breakingnc 

't.vyo  |-|VI  -  lO  /-%pi  C.\J  lO 
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3:24  PM-15AP 

3,859  RETWEETS  81  FAVORITES 
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1,170  RETWEETS 

77  FAVORITES 
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“  The  Boston  Marathon  bombings  are  certainlyi 
a  tale  of  terror,  but  also  a  tale  about  the  power 
and  perils  of  social  media.”  -  cbsnews.com 


By  (S)NewsReputation,  MRI  Universidad  de  Navarra 


C2  and  Citizen  Engagement 

•  Nationwide  Suspicious  Activity  Reporting  (SAR) 
Initiative  (NSI) 

►  Oneway  portal  for  posting  and  compiling  anonymous  reports 
of  suspicious  activity 

►  Does  not  support  the  kind  of  real-time  interaction  that 
occurred  in  the  Boston  Marathon  event 

•  Social  media  can  support  real-time  collaboration 

•  Need  to  adapt  command  and  control  systems  and 
processes  to  exploit  technologies  for  communicating 
directly  with  citizens 

►  Design  and  evaluate  new  systems  and  processes 

►  Achieve  benefits  whiie  mitigating  problems 

►  Train  operators  in  new  systems  and  processes 


Policy  Directives 

•  Presidential  Policy  Directive-8  (PPD-8)  states:  “Our  national 
preparedness  is  the  shared  responsibility  of  all  levels  of  government, 
the  private  and  nonprofit  sectors,  and  individual  citizens.  Everyone 
can  contribute  to  safeguarding  the  Nation  from  harm..” 

•  National  Strategic  Narrative  calls  for  diverse  and  deployable  Inter 
Agency,  and  a  well-informed  and  supportive  citizenry.  * 

•  National  Capital  Region  Homeland  Security  Strategic  Plan  calls  for 
sharing  information  needed  to  make  informed  and  timely  decisions; 
take  appropriate  actions;  and  communicate  accurate,  timely 
information  with  the  public. 

•  Department  of  Defense  Quadrennial  Defense  Review,  dated 
February  2010,  identified  defending  the  homeland  and  support  to  civil 
authorities  as  one  of  6  key  missions  in  which  the  Department  must 
further  rebalance  policy,  doctrine  and  capabilities 
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*  Monograph  from  Woodrow  Wilson  Center  for 
5  Scholars  authored  by  former  members  of  the  Joint 


Hypothesis 

•  Real-time  interaction  of  citizens  in  planning  and  execution 
of  a  military/civilian  contingency  operation  would  improve 
its  result 

•  Crowdsourcing  technology  is  a  viable  method  of  including 
American  citizens  in  the  decision-making  process 

•  Testing  the  hypothesis: 

►  Implement  prototype  system  to  employ  crowdsourcing  for  citizen 
participation 

►  Simuiate  crisis  in  which  civiiian/miiitary  emergency  managers  use 
system  to  interact  with  a  cross-section  of  the  American  pubiic 
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SIMEX 


•  MITRE  National  Security  Experimentation  Laboratory  (NSEL) 

►  Conducts  simulation  3-5  simulation  experiments  (SIMEXs)  per  year  to  examine 
C4ISR  processes  in  support  of  ground,  maritime,  space  and  air  operations 

►  Use  real  operators,  real  C4ISR  systems,  simulated  scenario  and  reports 

►  42  SIMEXs  conducted  since  2002 

•  SIMEXs  support  multiple  sponsors  to  examine: 

►  Tactics,  techniques  and  procedures  (TTPs) 

►  Concept  of  operations  (CONORS) 

►  Interoperability  requirements 
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Simulated 


Equipment 


Real 


Monitoring  Map 

Systems  Displays 

W  Communication 
I  Devices 

■  Humans-in-the- 
I  loop 
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Testing  the  Hypothesis:  A  SIMEX  examining 
Citizen  Participation  in  Crisis  Response 

Primary  Goal:  Examine  impact  of  citizen  involvement  on 

tactical/operational  decision-making  and  implementation. 

Objectives: 

►  Refine  and  evolve  CONORS  and  TTPs  for 
citizen  participation  in  tactical/operational 
planning  and  implementation 

►  Refine  and  evolve  prototype  DSS 

►  Examine  impact  of  DSS  on  tactical/operational  decision-making  &  execution. 

Scenario:  Defense  Support  of  Civil  Authorities 

►  Radiological  Dispersal  Device  detonates  on  George  Mason  University 
campus. 

►  Notional  NCR  military/civilian  emergency  managers  collaborate  from 
Emergency  Operations  Center  (emulated  at  the  NSEL  at  MITRE  McLean) 

►  Student  volunteers  from  George  Mason  University  use  DSS  to  collaborate  in 
msponse  decision  making. 


Citizens’  Emergency  Response  Portal 
System  (CERPS)  Public  Interaction 


Simulated  Sensory 
Environment  (SSE) 


Citizens’  Emergency 
Response  Portal  (CERP) 


Unfolding 
experiment 
events 
(view) 


Reported 

events 

(view  and  post) 


Chirp  [open-source  Twitter  clone] 


Citizens 


(GMU  students) 


News 


Simulated 
News  Network 


Discussion 
of  events 
(view  and  post) 
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“Chirps” 
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Polling 


C*\  Center 


Citizens’  Emergency  Response 

Portal  (CERP) 


Based  upon  Ushahidi  platform 
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SSE:  Participants’  View  of  Scenario 


Visualization: 

•  A  live  video  stream  of  the  3D 
virtual  environment 


A  2D  map  with  notable  locations} 


•  A  virtual  camera  to  take  pictures 
and  share  with  others 


Simulation  of  other  senses: 
Descriptions  of  auditory,  tactile 
and  olfactory  sensations 
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Chirp 


•  Twitter-like  application  for 
providing  brief  messages  and 
responding  to  opinion  polls 

•  Citizens  share  observations 
and  connect  with  others 

>  Operators  can  join  the 
discussion 

•  Operators  can  poll  the  citizens 
to  obtain  information 
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CERPS  SIMEX 

Objective: 

►  Examine  impact  of  CERPS  and  citizen  involvement  on 
tactical  /  operational  decision-making  and  execution 

Participants: 

►  Emergency  management  personnel:  national  (DoD,  FBI,  DHS, 
National  Guard),  state,  county,  city,  and  university 

►  GMU  student  volunteers  to  play  role  of  citizens 

Experiment: 

►  Simulate  crisis 

►  Execute  crisis  procedures 

►  Students  interact  with  responders  via  CERPS 

Evaluation: 

►  Examine  results  on  metrics  of  interest 


CERPS  SIMEX  Operational  View 


Students 
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SIM  EX  Process 


Experimentation  plan;  Data 

Experiment  data; 

Collection  and  Analysis 

Daily  After  Action  Reviews 

Plan  (DCAP);  Simulation 

architecture;  Network 

architecture 

Concept  paper; 
Scenario  event  list; 
Sequence  diagram; 
Scenario  walkthrough 
questions/issues/gaps; 
Scenario  process  model 


Test  plans; 

Test  reports; 

Problem  report  tracking/resolution 


Quick  Look  Report 
Final  Report; 
Sponsor  briefs 


^  -  Initial  Planning  Conference  (IPC) 
^  -  Final  Planning  Conference  (FPC) 
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GMU  Tasks 


•  Advise  on  CONORS 

•  Coordinate  IRB  approval 

•  Recruit  student  participants 

•  Support  training 

•  Coordinate  strategic  communications  plan 
with  MITRE  community  relations  (avoid  “war 
of  worlds  effect”) 

•  Participate  in  EOC 


Student  Participation 

Participants 

►  Goal:  200  student  participants 

►  Actual:  199  recruited,  125  trained,  114  participated 

►  Paid  $95  in  Mason  money  plus  iPad  for  top  performer 

Activities: 

►  Training  session  (2  hrs) 

►  Test  runs  (2  hrs) 

►  Experimental  sessions  (at  least  5  hrs) 

-  A  different  virtual  emergency  each  day  for  5  days 

-  Respond  to  virtual  environment  through  CERPS 

-  Minimum  of  5  hours 

►  Feedback  session  (no  more  than  30  min) 


Government  Stakeholders 


•  NORTHCOM 

•  Joint  Staff 

•  Fairfax  County 

•  Virginia  Commonwealth 

•  DHS/FEMA 

•  National  Guard  Bureau 

•  FBI 

•  Israeli  Home  Front  Command* 
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Timeline 


•  Summer  2012: 

►  Develop  concept  of  operations,  scenario, 
data  collection  and  analysis  plans 

►  Obtain  IRB  approval 

►  Develop  publicity  plan 


•  September  2012: 

►  Recruit  and  train  participants 

•  October  201 2: 

►  Conduct  SIMEX  (Oct  1  -5) 

►  Produce  quick-look  briefing 

•  November  2012: 


►  Release  report  to  public 

I 


O'*  I  Center 


UNIVERSITY 
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CERPS  SIMEX  Command  and  Control 


•  Variety  of  cells  within  a  unified  Emergency  Operations  Center 


(EOC) 

•  County  EOC  Commander  is  in  charge  of  overall  management 


•  Incident  Commander  (County  Fire  Chief)  controls  on-scene 
response  from  Incident  Command  Post  (ICP) 

•  EOC/Responder  command  and  control,  perceived  situational 
awareness  and  notional  response  operations  are 
emulated/simulated  at  NSEL 


•  Public  represented  by  GMU  Student 
Volunteers  operating  from  Campus 


O'*  I  Center 
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Scenario 


Initial  Conditions 

►  Boy  Scout  Jamboree  (National  Special  Security  Event)  taking  place  in  area 

EOC  stood  up,  Federal,  NG  CERP-T,  CST  in  place;  NG  units  on  standby  in  Fairfax 
county  for  crowd  control,  checkpoints,  security,  etc. 

►  Rally  at  the  Johnson  Center  to  protest  controversial  author  Simon  Pierce, 
PhD.  who  is  speaking  to  a  packed  room  at  the  GMU  Johnson  Center  (g)  1800 

►  Fairfax  County  and  GMU  websites  hacked  by  Anti-Pierce  Group  and 
replaced  with  messages  threatening  violence 

►  Sold  out  concert  taking  place  at  Patriot  Center 

Emergency  Events 

►  Confrontations  between  protestors  and  rally  attendees 

►  Anti-Pierce  Group  detonates  vehicle  bomb  containing  radiological  device  on 
campus 

►  Anti-Pierce  Group  detonates  explosive  backpacks  in  crowds  at  Johnson 
Center 

►  Additional  secondary  detonations  and  threats  of  additional  attacks  take  place 
throughout  the  run 

All  runs  were  variants  of  this  basic  scenario  with  times,  locations,  and 
magnitudes  modified 


SIMEX:  Emergency  Operations  Center 


01  EXCON 
02  Data  Collection 
Lead 

03  Scenario  Lead 
04  SIM  Control 
05  Integration  Lead 
06  Development 
Lead 

07  Decision  Support 
Lead 

08  Tech  Support 
09  SSE  Lead 

10  Incident 
Commander/ 

Campus  Police 

1 1  News  Media 

12  DCO 
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State  Cell 


13  CERP  Administrator 

15City  EDMSIM 

16  JTFEDMSIM 

17  NGB  EDMSIM 
19  FBI  EDMSIM 


20  Campus  Cell 
Commander  /  EDMSIM 

30  City  Cell  Commander  / 
PR 

40  County  EOC 
Commander  /  EDMSIM 

41  County  PR 

42  WebEOC  Controller 


50  State  Cell  Commander/ 
PR /EDMSIM 

52  JTF-NCR  LNO 

53  State  NG  SEPLO 

54  FEMA  FCO 

55  FEMA  External  Affairs 

56  JTF-CS  LNO 

60  FBI  SAC 

61  FBI  PR 

62  FBI  CTOC  Coordinator 


County  Cell  City  Cell 


Media  Attention 

Experiment  Crowdsources  Public  in  Emergency  Response 
Decision-Making 

►  http://www.hstodav.us/industrv-news/aeneral/single-article/experiment- 

crowdsources-public-in-emergencv-response-decision- 

makina/9e632d951  b75fa299ac746a4ce2d55df.html 

This  is  just  a  test:  Emergency  responders  tap  the  Twitterverse 

►  http://www.nextaov.eom/emeraina-tech/2012/10/just-test-emeraencv- 

responders-tap-twitterverse/58622/?oref=ng-HPtopstory 

Mason  Students  Observe  and  Report  During  Mock  Attack  in 
Fairfax 

►  http://about.amu.edu/mason-students-observe-and-report-durina-mock- 

attack-in-fairfax/ 

Safety  Tweet:  Northern  Virginia  Magazine  by  Jenna  Makowski 
January  14,  2013 

►  http://www.northernvirainiamaa.com/buzz-bin/2013/01/15/safetv-tweet/ 


Student  Perceptions 
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60.00% 

50.00% 

40.00% 
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20.00% 
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The  CERP  application  I  communicated 
kept  me  informed  several  thoughts 
about  the  simulated  through  the  Chirp 
emergency.  application. 


I  found  the  Chirp  I  felt  that  decision 

messages  from  the  makers  were  taking 
Emergency  my  contributions 

Operations  Center  into  account. 
(EOC)  to  be  useful. 


■  Strongly  agree 

■  Agree 

■  Neither  agree  nor  disagree 

■  Disagree 

■  Strongly  disagree 

■  Not  Applicable 


Chirp  and  Poll  Usage 


UNIVERSITY 


Usage  steadily  increased  through  Day  3 
Unpianned  outage  on  Day  4 
Unplanned  interruption  on  Day  5 
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Student  Ratings:  Usefulness  and  Quality 


70.00% 


The  CERP  application 
kept  me  informed 
about  the  simulated 
emergency. 


I  communicated 
several  thoughts 
through  the  Chirp 
application. 


I  found  the  Chirp 
messages  from  the 
Emergency 
Operations  Center 
(EOC)  to  be  useful. 


I  felt  that  decision 
makers  were  taking 
my  contributions 
into  account. 


■  Strongly  agree 

■  Agree 

■  Neither  agree  nor  disagree 

■  Disagree 

■  Strongly  disagree 

■  Not  Applicable 


Figure  12.  Student  Ratings  of  Usefulness  of  CERPS 
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60.00% 

50.00% 

40.00% 

30.00% 

20.00% 

10.00% 

0.00% 


The  scenarios  seemed 
believable  to  me  -  I  feel 
that  they  could  have 
actually  happened. 


The  pictures  of  the 
simulated  emergency  were 
convincing 


The  descriptions  that  came 
with  the  pictures  of  the 
simulated  emergency 
allowed  me  to  understand 
what  I  was  seeing,  hearing, 
feeling,  and  smelling. 


■  Strongly  agree 

■  Agree 

■  Neither  agree  nor  disagree 

■  Disagree 

■  Strongly  disagree 

■  Not  Applicable 


Figure  13.  Quality  of  the  Virtual  Environment 
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Operator  T  rust 


Trust  in  Social  Media 


Completely  Somewhat  Neither  Somewhat  Completely 
Untrusted  Untrusted  Trusted  nor  Trusted  Trusted 

Untrusted 


Operators  showed  trust  in  social  media 

Influence  of  simulated  “bad  actors”  was  limited  and  short  term 

Trust  would  have  been  improved  with  geospatial  information 
(disabled  for  privacy  reasons) 


Realism 


Realism  of  CERP  and  Chirp  Posts 


■  Realism  of  CERP  and  Chirp 
Posts 


Operators  reported  interactions  with  public  felt  authentic 
and  added  valuable  dimension  to  experiment 

Operators  reported  missing  public  interaction  during 
unplanned  run  without  interaction 


Conclusions 

Demonstrated  potential  for  positive  impacts  from 
citizen  interaction  with  emergency  managers 

►  Augment  91 1  -type  information  about  incidents 

►  Sentiment  analysis  of  social  media  traffic 

Helped  emergency  managers  understand  mood  of  public 

-  Allowed  managers  to  adjust  communications  strategies  to  better  respond  to 
needs  of  public 

Highlighted  challenges  of  public  interaction  through 
social  media 

►  Vet  information  for  accuracy 

►  Account  for  possible  influence  of  bad  actors 

►  Mitigate  potential  for  emergency  managers  to  be  distracted 
by  vocal  social  media  users 


Research  Issues 


•  Effective  integration  of  citizen  input  into  C2  processes 

•  Integrating  public  participation  into  logistics 

►  Provide  timely  assistance  where  it  is  most  needed 

•  Human  factors  -  citizens  and  operators 

•  Identifying  trusted  sources  /  filtering  bad  information 

•  Information  security 

•  Providing  information  to  operators  while  protecting 
personally  identifiable  information 

•  Mining  large  volumes  of  social  media  for  actionable 
information 
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Policy  Issues 

•  Expectations  -  If  we  start  responding  to  Chirp,  does  that  establish  a  public 
expectation  that  we  will  always  respond  to  Chirp  (especially  for  911 -type 
Chirps)? 

•  Liability -\Nha{  happens  if  you  ask  public  to  do  something  (like  evacuate 
using  a  certain  route)  and  they  get  hurt  as  a  result? 

•  Privacy  -  How  must  personal  info,  geolocations,  etc.,  be  handled? 

•  Two  wav  communications  with  public,  followinQ/likinQ  -  Who  will  be  allowed 
to  do  this  and  under  what  circumstances? 

•  Law  Enforcement  -  Emergency  management  has  fewer  constraints  on 
interactions  than  law  enforcement 

•  Consent  -  Does  the  public  need  to  consent  before  we  respond  using  social 
media? 

•  Involvement -\Nho  is  the  public?  Who  can  be  involved? 


Masow 
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Current  policies  were  not  designed  with  social  media  in  mind  and  will  need  XG 

be  evolved  to  enable  CERPS-like  capabilities 


O'*!  Center 


Proposed  Follow-On  SIMEX 


•  Maintain  theme 

•  Include  additional  stakeholders 

•  Follow  similar  planning  and  execution  schedule 

•  Incorporate  alternative  tools  as  appropriate  from 
government  and  industry 

•  Expand  to  include  GMU  campus  and  surrounding 
region  (“College  Town  USA”) 


*■  Larger  population  sample 


*■  Students,  staff  and  faculty 
►  Other  participants  from  community 

•  Expand  /  revise  EOC  staffing 
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T  ake  Aways 


•  The  cloud  and  social  media  bring  major  new 
opportunities  for  decision  support  in  crisis 
situations 


•  We  are  just  beginning  to  understand  how  to  exploit 
these  opportunities 


•  CERPS  SIMEX  was  an  important  first  step  in 


adapting  C2  processes  and  tools  to  new 

•  Additional  work  is  needed  to  improve  our 
understanding  of  issues  and  solutions 

*■  Follow  on  SIMEXs 

►  Research  on  technology,  tools,  processes 
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t (policy  analysis  and  development 

C^IQ^Tter 


Thank  You! 


GMU 

•  Stu  Wharton  (participant 
coordinator) 

•  Dave  Farris  (emergency 
management) 

•  Paul  Liberty  and  Jim 
Greif  (public  relations) 

•  George  Ginkovsky 
(university  police) 


MITRE 

•  Jim  Dear  (Project 
Lead,  NSEL) 

•  Jackson  Ludwig 

•  Jennifer  Mathieu 

•  Alaina  McCormack 

•  Tobin  Bergen-Hill 

•  Karina  Wright 

•  ...  and  many  others 
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and  all  the  SIMEX  participants 
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